Neural Networks as an Alternative to Photogrammetry. Using Instant NeRF and Volumetric Rendering

The use of artificial intelligence (AI) and machine learning algorithms has recently revolutionized many areas, including heritage enhancement through the Digital Twin and its use in the VR dimension. The paper discusses the potential use of various artificial intelligence techniques the Generative Adversarial Networks (GAN) in the development of interactive and immersive VR serious games for heritage enhancement. GANs can be used to generate realistic and fully artificial images of spaces, objects and human faces, which can be applied in the design and concept process for building interactive environments. In addition, machine learning algorithms can improve the adaptability of the game to the user, for example by automatically structuring the difficulty level or plot of the game based on the user’s actions. In recent years, however, new, higher-performance artificial intelligence processes have emerged that have surpassed the generative capabilities of GAN algorithms, the latter of which were mainly used to produce 2D images of nonexistent elements. NeRF (Neural Radiance Field) is a technology developed by NVIDIA supported by a complex neural network recently optimized by a proprietary code called Instant NeRF. This technology enables the rapid and accurate generation of detailed 3D models of physical environments using much less photographic data than alternative photomodeling techniques that do not directly support artificial intelligence. In the future, this technology can be used to generate virtual environments that are indistinguishable from reality and can also be actively used for heritage enhancement.